docs: correct stable row ID migration and storage - #8852
docs: correct stable row ID migration and storage#8852lance-gatefixer[bot] wants to merge 80 commits into
Conversation
|
Important Format specification voteThis PR modifies the Lance format specification, so it requires 3 binding +1 votes from PMC members (excluding the proposer) and a minimum 72-hour voting period, weekends excluded, before it can merge. Vote by approving this PR (+1) or requesting changes (−1, a veto). See the voting process. Status: ❌ Blocked — 0 of 3 required approvals
Updated automatically by the format-spec vote gate, which re-checks every 15 minutes — just voted? Re-check now (press Run workflow; leave the input blank to re-check every open format PR). A PMC member may apply the |
|
Blocked: the current head cannot pass Remote head |
|
Blocked: the current head cannot pass Remote head |
|
Blocked: the current head is held by an unrelated Windows runner failure and the required format-spec approval gate. Remote head |
|
Blocked: the current head remains held by an unrelated Windows hosted-runner failure and the required format-spec approval gate. Remote head |
|
Blocked: the current head cannot satisfy Remote head |
|
Blocked: the current head cannot satisfy format-spec-vote until the required PMC approval process completes. Remote head 01197a5 contains the current main tip 085354e. The live format-spec-vote status targets Actions run 33217104161 and reports 0/3 PMC approvals on this commit; its evaluator job 99003082863 completed successfully, and all 32 current-head CheckRuns pass. I fetched both branches, verified ancestry and remote identity, inspected the current status and evaluator logs, and ran git diff --check; no attributable code defect remains to repair. The smallest next action is for three eligible PMC members to provide binding +1 approvals after the required voting period; alternatively, a PMC member may apply format-waived if this documentation correction qualifies as trivial. |
|
Blocked: the current head cannot satisfy format-spec-vote until the required PMC approval process completes. Remote head 01197a5 contains the current main tip 085354e. The live format-spec-vote status targets Actions run 33218990261 and reports 0/3 PMC approvals on this commit; its evaluator job 99008845859 completed successfully, and all 32 current-head CheckRuns pass. I fetched both branches, verified ancestry and remote identity, inspected the current status and evaluator logs, and ran git diff --check; no attributable code defect remains to repair. The smallest next action is for three eligible PMC members to provide binding +1 approvals after the required voting period; alternatively, a PMC member may apply format-waived if this documentation correction qualifies as trivial. |
|
Blocked: the current head cannot satisfy format-spec-vote until the required PMC approval process completes. Remote head 01197a5 contains the current main tip 085354e. The live format-spec-vote status targets Actions run 33219934264 and reports 0/3 PMC approvals on this commit; its evaluator job 99011652281 completed successfully, and all 32 current-head CheckRuns pass. I fetched both branches, verified ancestry and remote identity, inspected the current status and evaluator logs, and ran git diff --check; no attributable code defect remains to repair. The smallest next action is for three eligible PMC members to provide binding +1 approvals after the required voting period; alternatively, a PMC member may apply format-waived if this documentation correction qualifies as trivial. |
|
Blocked: the current head cannot satisfy format-spec-vote until the required PMC approval process completes. Remote head 01197a5 contains the current main tip 085354e. The live format-spec-vote status targets Actions run 33220685156 and reports 0/3 PMC approvals on this commit; its evaluator job 99013887195 completed successfully, and all 32 current-head CheckRuns pass. I fetched both branches, verified ancestry and remote identity, inspected the current status and evaluator logs, and ran git diff --check; no attributable code defect remains to repair. The smallest next action is for three eligible PMC members to provide binding +1 approvals after the required voting period; alternatively, a PMC member may apply format-waived if this documentation correction qualifies as trivial. |
|
Blocked: the current head cannot satisfy Remote head |
|
Blocked: the current head cannot satisfy Remote head |
|
Blocked: PR #8852 at remote head Remote head I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, inspected the live checks, logs, reviews, inline threads, and PR conversation, passed whitespace, Rust formatting, protobuf compilation, and MkDocs validation, and pushed the assigned branch normally; it was already up to date. No attributable repository defect remains to repair, so no speculative code change was made. The smallest next action is for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Remote head I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, inspected the live checks, logs, reviews, inline threads, and PR conversation, passed whitespace, Rust formatting, protobuf compilation, and MkDocs validation, and pushed the assigned branch normally; it was already up to date. No attributable repository defect remains to repair, so no speculative code change was made. The smallest next action is for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The pull-request patch remains unchanged. The new base change only removes a source commit-handler override from new-dataset clone commits; stable-row-ID migration still commits a Merge against an existing dataset through the unchanged transaction path. Index rebasing and sequence serialization are untouched, so the corrected migration and inline/external storage guidance remains accurate.
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, then pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are to rerun the failed Rust job and obtain three eligible PMC +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, then pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are to rerun the failed Rust job and obtain three eligible PMC +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, then pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are to rerun the failed Rust job and obtain three eligible PMC +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, inspected the live checks and logs, and passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build. I pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, inspected the live checks and logs, and passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build. I pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready PR identity, inspected the live checks and logs, and passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build. I pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live checks, logs, reviews, inline comments, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live checks, logs, reviews, inline comments, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live checks and logs, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. No attributable repository change is justified by the available evidence. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and the exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. The available evidence does not justify an attributable repository change. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. The available evidence does not justify an attributable repository change. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. The available evidence does not justify an attributable repository change. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, whitespace checks, Rust formatting, protobuf compilation, and MkDocs build, and pushed the assigned branch normally; it was already up to date. The available evidence does not justify an attributable repository change. The smallest next actions are for a maintainer to rerun the failed Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Alternatively, an authorized PMC member may apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, Rust formatting, protoc-enabled package check, whitespace check, and MkDocs build, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, logs, and PR conversation, passed the focused Rust test, Rust formatting, protoc-enabled package check, whitespace check, and MkDocs build, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply |
|
Blocked: the current head remains held by a non-reproduced macOS Rust test failure and the required format-spec approval gate. Remote head |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, and logs, passed the focused Rust test and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, and logs, passed the focused Rust test and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline comments, checks, and logs, passed the focused Rust test and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS Rust job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply |
|
Blocked: PR #8852 at remote head Current I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the live issue, reviews, inline threads, conversation, checks, and logs, passed the exact Rust test, protobuf compilation, MkDocs build, and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply |
|
Blocked: PR #8852 at remote head b3f7e10 remains blocked by a non-reproduced macOS-only ANN test failure and the required format-spec-vote approval gate. Current main tip 09168e9 is an ancestor of the verified remote repair head. Rust run 33793005054, mac-build (stable) job 100773858654, failed only the stable-storage, stable-row-ID, fast-mode IVF-HNSW test_ann_prefilter case with "Vector column vector not found in batch"; 1,372 other tests passed before fail-fast, including adjacent variants, and the Linux and Windows Rust jobs passed. This repair changes only the row-ID format specification and protobuf comments, while the exact failing case passes locally with the CI profile and feature set, so the available evidence does not justify an attributable code change. Separately, format-vote run 33808380522, evaluator job 100824185512, completed successfully but reported "PR #8852: 0/3 PMC approvals on this commit". I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the current checks and logs, passed the exact Rust test, protobuf compilation, MkDocs build, and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply format-waived; a deterministic rerun failure with new logs would support further diagnosis. |
|
Blocked: PR #8852 at remote head b3f7e10 remains blocked by a non-reproduced macOS-only ANN test failure and the required format-spec-vote approval gate. Current main tip 09168e9 is an ancestor of the verified remote repair head. Rust run 33793005054, mac-build (stable) job 100773858654, failed only the stable-storage, stable-row-ID, fast-mode IVF-HNSW test_ann_prefilter case with "Vector column vector not found in batch"; 1,372 other tests passed before fail-fast, including adjacent variants, and the Linux and Windows Rust jobs passed. This repair changes only the row-ID format specification and protobuf comments, while the exact failing case passes locally with the CI profile and feature set, so the available evidence does not justify an attributable code change. Separately, format-vote run 33809686704, evaluator job 100828348787, completed successfully but reported "PR #8852: 0/3 PMC approvals on this commit". I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the current checks and logs, passed the exact Rust test, protobuf compilation, MkDocs build, and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply format-waived; a deterministic rerun failure with new logs would support further diagnosis. |
|
Blocked: PR #8852 at remote head b3f7e10 remains blocked by a non-reproduced macOS-only ANN test failure and the required format-spec-vote approval gate. Current main tip 09168e9 is an ancestor of the verified remote repair head. Rust run 33793005054, mac-build (stable) job 100773858654, failed only the stable-storage, stable-row-ID, fast-mode IVF-HNSW test_ann_prefilter case with "Vector column vector not found in batch"; 1,372 other tests passed before fail-fast, including adjacent variants, and the Linux and Windows Rust jobs passed. This repair changes only the row-ID format specification and protobuf comments, while the exact failing case passes locally with the CI profile and feature set, so the available evidence does not justify an attributable code change. Separately, format-vote run 33811058393, evaluator job 100832729570, completed successfully but reported "PR #8852: 0/3 PMC approvals on this commit". I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the current checks and logs, passed the exact Rust test, protobuf compilation, MkDocs build, and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply format-waived; a deterministic rerun failure with new logs would support further diagnosis. |
|
Blocked: PR #8852 at remote head b3f7e10 remains blocked by a non-reproduced macOS-only ANN test failure and the required format-spec-vote approval gate. Current main tip 09168e9 is an ancestor of the verified remote repair head. Rust run 33793005054, mac-build (stable) job 100773858654, failed only the stable-storage, stable-row-ID, fast-mode IVF-HNSW test_ann_prefilter case with "Vector column vector not found in batch"; 1,372 other tests passed before fail-fast, including adjacent variants, and the Linux and Windows Rust jobs passed. This repair changes only the row-ID format specification and protobuf comments, while the exact failing case passes locally with the CI profile and feature set, so the available evidence does not justify an attributable code change. Separately, format-vote run 33813413195, evaluator job 100840166465, completed successfully but reported "PR #8852: 0/3 PMC approvals on this commit". I fetched both assigned refs, verified current-base ancestry and exact Ready-PR identity, inspected the current checks and logs, passed the exact Rust test, protobuf compilation, MkDocs build, and diff checks, and pushed the assigned branch normally; it was already up to date. The smallest next actions are for a maintainer to rerun the failed macOS job and for three eligible PMC members to provide binding +1 approvals on this exact commit. Practical alternatives are rerunning the full Rust workflow and, if this documentation correction qualifies as trivial, having an authorized PMC member apply format-waived; a deterministic rerun failure with new logs would support further diagnosis. |
Summary
Root cause
The format documentation and protobuf comments still described pre-implementation plans: they said existing datasets could not migrate and presented a 200 KB inline/external threshold as active writer behavior. Migration has since shipped, but writers still emit every sequence inline and readers cannot load external version sequences.
Validation
protoc --proto_path=../protos --descriptor_set_out=/dev/null ../protos/table.protouv run mkdocs build(completed with pre-existing missing-link warnings outside this change)git diff --checkFixes #8851